Reusing Monolingual Pre-Trained Models by Cross-Connecting Seq2seq Models for Machine Translation

This work uses sequence-to-sequence (seq2seq) models pre-trained on monolingual corpora for machine translation. We pre-train two seq2seq with the source and target languages, then combine encoder of language model decoder model, i.e., cross-connection. add an intermediate layer between to help mapping each other since modules are completely independently. These can as a multilingual because one be cross-connected another any language, while their capacity is not affected by number languages. will demonstrate that our method improves translation performance significantly over random baseline. Moreover, we analyze appropriate choice layer, importance part change along size bitext.